Evaluation of a Wireless Handheld Medication Management Device in the Prevention of Drug-Drug Interactions in a Medicaid Population

BACKGROUND: With the passage of the Health Information Technology for Economic and Clinical Health (HITECH) Act, widespread adoption of certain health information technologies, such as electronic health records (EHRs) and electronic prescribing (e-prescribing), is imminent. Drug-drug interaction (DDI) screening and medication history information are commonly incorporated into health information exchange systems to improve medical decision making, safety, and quality of care, but the value of these features is unclear. OBJECTIVES: To evaluate the effect of providing access to an early generation electronic medication management program with medication history accessible to prescribers via a wireless handheld personal digital assistant (PDA) device on the incidence of potential DDIs (i.e., DDIs that may or may not cause patient harm). METHODS: This study employed a retrospective pre-intervention/post-intervention study design with a comparison group to evaluate the effectiveness of a wireless handheld medication management program in preventing serious potential DDIs. Licensed prescribers in a state Medicaid program who wrote prescriptions during the period from August 2003 through June 2006 were included in this study. The intervention (PDA) group consisted of clinicians who requested and were granted access to the wireless handheld device containing prescription drug history between August 1, 2004, and June 30, 2005. Initially the device contained 100-day patient-specific medication history, but other functionalities were added during the study period including the ability to check for drug-drug interactions and e-prescribing. The comparison group consisted of prescribers who sent a request to obtain, but did not receive, the wireless handheld device during the same time period. Baseline prescribing patterns of 25 previously identified clinically important potential DDIs were assessed over two 12-month periods, one period prior to (baseline) and one period after (follow-up) an index date (date of device deployment for PDA group; date of request for comparison group). A random-effects negative binomial model was used to analyze the primary outcome, the number of potential DDIs per prescriber per 12-month time period. A secondary outcome of interest, the likelihood that a prescriber would prescribe at least 1 potentially interacting medication pair during the baseline and follow-up periods, was analyzed using a random-effects logistic model. RESULTS: A total of 1,615 prescribers constituted the PDA group, and 600 prescribers made up the comparison group. Prescribers in the 2 groups were significantly different in their specialty practice areas (P  less than  0.001), number of pharmacy claims at baseline (P less than  0.001), and the likelihood of prescribing at least 1 potential DDI combination during the 1-year baseline period (P = 0.003). However, the prescriber groups were similar in their average age (P = 0.241) and geographic location (P = 0.181). The most widely prescribed potential DDIs included those involving warfarin with nonsteroidal anti-inflammatory drugs (NSAIDs) and thyroid hormones. The median number of patient medication history updates requested per PDA group prescriber during follow-up was 24 (range 0 to 1,073). At baseline, 1,104 (68.4%) of the PDA group and 449 (74.8%) of the comparison group had no potential DDIs. During the next year, 1,131 (70.0%) and 462 (77.0%) of the PDA group and comparison group, respectively, had no DDIs. The incidence rate ratio was 1.01 (95% CI = 0.87-1.17) for the PDA group relative to the comparison group for change in number of potential DDIs. In the logistic regression model, the odds of prescribing at least 1 potential DDI did not significantly differ by group (odds ratio = 1.26, 95% CI = 0.96-1.66). These results indicate that there was no significant difference between the intervention and comparison group with regard to the change in the rate of potential DDIs between the baseline and follow-up periods. CONCLUSIONS: A stand-alone medication management program in a wireless PDA device was not frequently used by most prescribers to update patient medication histories and was not associated with a reduction in the rate of prescribing potentially clinically important DDIs.

OBJECTIVE: To evaluate the effect of providing access to an early generation electronic medication management program with medication history accessible to prescribers via a wireless handheld personal digital assistant (PDA) device on the incidence of potential DDIs (i.e., DDIs that may or may not cause patient harm).
METHODS: This study employed a retrospective pre-intervention/postintervention study design with a comparison group to evaluate the effectiveness of a wireless handheld medication management program in preventing serious potential DDIs. Licensed prescribers in a state Medicaid program who wrote prescriptions during the period from August 2003 through June 2006 were included in this study. The intervention (PDA) group consisted of clinicians who requested and were granted access to the wireless handheld device containing prescription drug history between August 1, 2004, andJune 30, 2005. Initially the device contained 100-day patient-specific medication history, but other functionalities were added during the study period including the ability to check for drug-drug interactions and e-prescribing. The comparison group consisted of prescribers who sent a request to obtain, but did not receive, the wireless handheld device during the same time period. Baseline prescribing patterns of 25 previously identified clinically important potential DDIs were assessed over two 12-month periods, one period prior to (baseline) and one period after (follow-up) an index date (date of device deployment for PDA group; date of request for comparison group). A random-effects negative binomial model was used to analyze the primary outcome, the number of potential DDIs per prescriber per 12-month time period. A secondary outcome of interest, the likelihood that a prescriber would prescribe at least 1 potentially interacting medication pair during the baseline and follow-up periods, was analyzed using a random-effects logistic model. RESULTS: A total of 1,615 prescribers constituted the PDA group, and 600 prescribers made up the comparison group. Prescribers in the 2 groups were significantly different in their specialty practice areas (P < 0.001), number of pharmacy claims at baseline (P < 0.001), and the likelihood of prescribing at least 1 potential DDI combination during the 1-year baseline period (P = 0.003). However, the prescriber groups were similar in their average age (P = 0.241) and geographic location (P = 0.181). The most widely prescribed potential DDIs included those involving warfarin with nonsteroidal anti-inflammatory drugs (NSAIDs) and thyroid hormones. The median number of patient medication history updates requested per PDA group prescriber during follow-up was 24 (range 0 to 1,073). At baseline,

R E S E A R C H
• Dissemination of a handheld wireless device to download medication history to licensed prescribers in a state Medicaid program was not associated with a significant reduction in the rate at which 2 potentially interacting medications were prescribed relative to a comparison group comprising licensed prescribers in the same state Medicaid program who did not have access to the technology (P > 0.10). • The majority of prescribers (68.4% in the intervention group and 74.8% in the comparison group) did not prescribe any of the targeted potential DDI pairs during the baseline period.
providers in selecting the most appropriate drug for the patient; such features may include, but are not limited to, DDI screening, drug-allergy contraindications, laboratory results, medication history, and dosage alerts. A number of articles have reviewed the evidence of CPOE/CDS on medication safety. [15][16][17][18][19][20][21] Based on a systematic review of studies published through April 2006, Ammenwerth et al. (2008) concluded that electronic systems can diminish the risk of ADEs and medication errors; 20 however, most of the evidence to support this conclusion has come from studies conducted primarily in inpatient settings. Ammenwerth et al. advocated for stronger study designs, studies involving wider geographic and clinical settings, and studies involving commercially available systems to improve the evidence and generalizability of the potential safety benefits associated with these technologies. 20 We evaluated the effect of a wireless handheld personal digital assistant (PDA) medication management program, capable of providing physicians with nearly real-time access to patientspecific medication histories integrated around comprehensive prescription drug information, on potential DDI medication errors. This study is unique in that it involves a medication management application available on a handheld electronic device, focuses on a specific type of preventable medication error (DDIs), and all Medicaid prescribers were potentially eligible to participate in the study regardless of practice setting. The objective of this analysis was to evaluate the effect of this wireless handheld medication management program on the incidence of potential DDIs, a type of preventable medication error.

Description of the Medication Management Program
This study evaluated a handheld personal digital assistant (PDA) device with which prescribers could download, via cellular networks, medication histories for patients who received 1 or more prescriptions authorized by the prescriber. The medication history included those medications ordered by the prescriber as well as all other medications that had been obtained by the patient from other prescribers under the state Medicaid program. The functionality of the device evolved over time from initially containing only medication histories to a device with e-prescribing and other clinical drug information. By the conclusion of the observation period, the system under evaluation in this study incorporated the following functionalities: (a) preferred drug list status information, (b) clinical drug information (e.g., clinical pharmacology, common adverse events, contraindications/precautions), (c) 100-day patient-specific prescription drug history, (d) alerts for drug-drug interactions, (e) refill histories, and (f) dose ranges for drugs prescribed outside of the dose ranges. E-prescribing functionality was not initially available and was added during the study time frame but was not extensively used by the providers. W ith the passage of the Health Information Technology for Economic and Clinical Health (HITECH) Act, widespread adoption of certain health information technologies (HIT), such as electronic health records (EHRs) and electronic prescribing (e-prescribing), is imminent. This act provides financial incentives for clinicians and hospitals to not merely adopt EHRs, but to make use of the technology in such a way that health care outcomes and processes are improved. 1 In order for eligible providers to qualify for government funds, a core set of objectives has to be met, including medication-related goals pertaining to e-prescribing, drug-drug interaction (DDI) checks, drug-allergy checks, and computerized provider order entry (CPOE) for prescriptions. 1 Medication safety is clearly a focus of the "meaningful use" objectives. The Institute of Medicine has urged the use of EHRs as an avenue for improving medication safety and quality. 2,3 It is estimated that approximately 1.5 million preventable adverse drug events (ADEs) occur annually in the United States. 2 Studies have indicated that ADEs in the ambulatory setting are common, but between 11% 4 and 28% 5 of these events are preventable.
DDIs are a type of preventable medication error. Each year, millions of patients are exposed to potential DDIs, 6,7 which may result in serious adverse events, including death. 8-11 DDI screening is a basic clinical decision support (CDS) feature incorporated into EHRs and e-prescribing systems to improve medical decision making, safety, and quality of care. 12 When 2 potentially interacting medications are prescribed for an individual, some DDI CDS screening programs will warn prescribers of the potential harm in the form of an automated alert or electronic message. DDI screening software can enhance a clinician's ability to identify clinically significant DDIs; research indicates that without the use of automated DDI alerts, prescribers' ability to recognize well-documented drug interactions is limited. 13,14 Health information technology often utilizes CDS to assist prescribers when selecting medications. Various features are often built in to medication management systems to assist • Of the clinically significant potential DDIs examined, the most commonly prescribed potentially interacting drug pair was warfarin and nonsteroidal anti-inflammatory drugs (NSAIDs), followed by warfarin and thyroid medications. • The medication management program allowed prescribers to access patient medication history; however, use of this feature varied. The median number of patient medication history updates requested per prescriber over a 12-month period in the intervention group was 24 (ranging from 0 to 1,073).

■■ Methods Study Design
This study employed a retrospective pre-intervention/postintervention study design with a comparison group to evaluate the effectiveness of a wireless handheld medication management program in preventing serious DDIs in a Medicaid population. The primary outcome was the number of serious potential DDIs detected through review of each health care practitioner's prescribing history for a single state's Medicaid population during a 1-year baseline and 1-year follow-up period. Thus, the analysis was conducted at the prescriber level, not at the patient level. A secondary outcome of interest was the likelihood of occurrence of at least 1 potential DDI for prescribers during the baseline and follow-up periods.

Participants
Licensed prescribers in a state Medicaid program who wrote prescriptions during the period from August 2003 through June 2006 were included in this study. The intervention group consisted of clinicians who requested and were granted access to the wireless handheld device between August 1, 2004, and June 30, 2005. To be included in the study, those providers who received the device had to keep it in their possession for at least 365 days. The comparison group was composed of prescribers who sent a request to obtain, but did not receive, the technology during the same time period. Higher prescription volume and prescriber residence within targeted geographic areas that permitted prescriber training were factors that increased access to the PDA device. For each provider, a 24-month window of pharmacy claims data was obtained-12 months prior to and 12 months after a specified index date. For the intervention group, the index date was defined as the date when the device was deployed to the provider. For the comparison group, the index date was the date of registration to obtain the wireless handheld device. Prescribers in the comparison group were excluded from the analysis if they were granted access to the device during the assessment period. Furthermore, any prescriber who failed to write at least 1 prescription for a state Medicaid patient in the 12-month period prior to the index date and the 12-month period following the index date was excluded from the analysis.

Data
Data to determine the rate of DDIs were obtained from pharmacy claims. The pharmacy claims dataset included the following fields: national drug code (NDC) number, date of service, quantity dispensed, days supply, and an indicator noting whether the prescription order was communicated to the pharmacy via the wireless handheld device. The data included all pharmacy claims from all prescribers for state Medicaid patients treated by prescribers in the PDA and comparison groups. Baseline prescribing patterns of 25 previously identi-fied clinically important potential DDIs were assessed over a 12-month period prior to the index date. 22 Follow-up prescribing patterns of potential DDIs were assessed over a 12-month period after the index date. The date of dispensing and days supply for each prescription were used to create an exposure window, and potential DDIs were identified if the exposure windows for drug combinations overlapped. Data collected on each prescriber included geographic region (urban or rural), age, and specialty. For each provider in the intervention group, the number of times patients' medication history was updated was also collected; it was assumed that the frequency with which patients' medication histories were updated reflected how often the prescriber was using the device.

Statistical Analysis
Descriptive statistics were used to summarize the characteristics of prescribers in the 2 study groups at baseline as well as patterns of prescribing potential DDIs in the 12-months before and after the index date. The Pearson chi-square test was used to examine differences in geographic location, prescriber specialty, the number of pharmacy claims at baseline, and the number of potential DDIs at baseline between the 2 groups. A 2-sample t-test was used to determine if there were significant differences in age between the intervention and comparison group. The Wilcoxon rank-sum test was used to determine whether the 2 groups differed in the median number of prescriptions at baseline and follow-up.
To test whether the wireless handheld medication device was associated with a decrease in the rate of prescribing potentially interacting drug pairs, a random-effects negative binomial regression model was fitted. The outcome of this model was the number of potential DDIs per prescriber per 1-year time period, measured in the 1-year baseline and follow-up periods. An interaction term between time period and group (PDA group or comparison group) was tested for inclusion in the model to assess whether the rate of change in number of potential DDIs differed by treatment group. The total number of pharmacy claims for medications ordered by each prescriber for individuals in the state Medicaid program for each time period of interest was included in the model as the exposure variable. The model also controlled for age of the prescriber, geographic location (urban/rural), time period, group (PDA group or comparison group), and specialty.
A second model, a random-effects logistic model, was fitted to assess whether the wireless handheld medication device was associated with a decrease in the likelihood of prescribing at least 1 potential DDI. The outcome, the presence of at least 1 potential DDI, was both time-dependent and binary. An interaction term between group (PDA group or comparison group) and time period was constructed to assess the rate of change in the likelihood of practitioners prescribing at least 1 potentially interacting drug pair. The model adjusted for the total number

Evaluation of a Wireless Handheld Medication Management Device in the Prevention of Drug-Drug Interactions in a Medicaid Population
of prescriptions written by each prescriber. The model also controlled for prescriber age, geographic location, time period, PDA group, and specialty.
Interaction terms were assessed at a significance level of 0.10. All other variables were considered statistically significant at alpha < 0.05. Sensitivity analyses using the same models, but with outliers removed, were performed to assess the robustness of the models. Outliers were defined as the 5 prescribers with the largest absolute differences in the rate of prescribing potential DDIs between baseline and follow-up periods. In addition, another sensitivity analysis re-examined the models without pediatric and psychiatry specialty data. To determine whether the frequency with which providers obtained medication histories from their PDA devices influenced results, models were also constructed in which PDAgroup prescribers were stratified by how often they updated patient medication histories. Likelihood ratio tests and Aikaike information criterion (AIC) values were used to help determine the most appropriate models.
Initial data cleaning and examination were done using SAS 9.1 (SAS Institute, Cary, NC). Analyses were performed using STATA 11 (STATA, College Station, TX). The study was approved by the University of Arizona Human Subjects Protection Program.

■■ Results
During the study period, 1,975 providers requested and obtained the wireless handheld medication device ( Figure 1). However, 303 of these participants returned the device within 365 days and were therefore excluded from the analysis. Of the 1,063 providers who requested and were initially denied access to the wireless handheld medication device, 424 were subsequently granted access to the device at a later time and were therefore excluded from the analysis. Of the remaining eligible study participants, 57 prescribers in the intervention group and 39 providers in the comparison group failed to prescribe at least 1 prescription during the 12-month period prior to the index date or the 12-month period following the index date and were therefore excluded from the analysis. The final study sample consisted of 1,615 prescribers in the intervention arm and 600 prescribers in the comparison arm.
Prescribers in the intervention and comparison groups were

Evaluation of a Wireless Handheld Medication Management Device in the Prevention of Drug-Drug Interactions in a Medicaid Population
Prescribers who failed to prescribe at least 1 prescription for a state Medicaid patient during either the baseline or follow-up period n = 39 Prescribers in the comparison group n = 600 Prescribers who failed to prescribe at least 1 prescription for a state Medicaid patient during either the baseline or follow-up period n = 57 PDA = personal digital assistant.

Flow Diagram Outlining Selection of Study Participants
Total number of prescribers requesting PDAs between August 2004 and June 2006 n = 3,038 Prescribers who were given immediate access to the PDA n = 1,975 Prescribers who were initially denied access to the PDA n = 1,063 Prescribers excluded because they did not keep the PDA in their possession for at least 365 days n = 303 Prescribers excluded because they were subsequently given access to the PDA during the study period n = 424 Prescribers who were initially denied the PDA and never had access to the technology during the study period n=639 Prescribers who were given immediate access to the PDA and kept the technology in their possession for at least 365 days n = 1,672 Prescribers in the PDA group n = 1,615 significantly different in their specialties, the number of pharmacy claims at baseline, and the presence of 1 or more potential DDIs at baseline. However, prescribers in both groups were similar in average age and geographic location ( Table 1). As a requirement to receive the device, the intervention group wrote significantly more prescriptions during both the baseline and follow-up periods compared with the comparison group (Table  2). Table 3  Prescribers with access to the wireless medication management program were able to access patient-specific medication histories (including prescriptions written by other prescribers). The number of patient medication history updates requested by prescribers increased gradually after adoption, reaching a peak of 8,397 updates per month in May 2005 ( Figure 2). Prescribers varied in their use of the system to access patient medication histories; the median number of patient medication history updates requested per prescriber in the intervention group was 24 (ranging from 0 to 1,073) during the 1-year follow-up period. On a monthly basis, the number of medication history updates varied dramatically over the course of the study, increasing from less than 0.02 updates per month to 11.7 updates per month 1 year after the study began. After peaking during the first half of the study period, use of the PDA appeared to decline and then stabilize over time. Over the course of the study, prescribers using the PDA submitted a total of 8,667 prescriptions electronically. The utilization rate of e-prescribing functional capability among PDA clinicians was relatively low; on average prescribers submitted 2      scribed during each period, prescriber age, geographic location, and specialty, there was no significant difference between the intervention and comparison group with regard to the rate at which 2 potentially interacting medications were prescribed between the baseline and follow-up periods ( Table 5). The interaction term was not significant at the P < 0.10 level and therefore dropped from the final model. The model indicated a statistically significant decline in the number of potential DDIs for both treatment groups between the baseline and follow-up period (incidence rate ratio [IRR] = 0.90, 95% confidence interval [CI] = 0.83-0.98, P = 0.019). Prescribers with rural practices had statistically significantly higher rates of potential DDIs compared with their urban counterparts in both time periods (IRR = 1.44, 95% CI = 1.18-1.76, P < 0.001). In addition, internal medicine providers were more likely to prescribe interacting drug pairs compared with family medicine practitioners in both time periods (IRR = 1.41, 95% CI = 1.21-1.66, P < 0.001). Prescribers with pediatric or psychiatry specialties were statistically less likely to prescribe these 25 potentially interacting drug pairs compared with family medicine practitioners (both P < 0.001). Stratifying the PDA group by frequency of prescriptions electronically for every 1,000 prescription claims. It was not possible to determine if prescribers were aware of potential DDIs as detected by the device because these data were not captured.

PDA = personal digital assistant.
Interactions involving the anticoagulant warfarin accounted for the majority of potential DDIs detected in this study ( Figure  3). The most widely prescribed potentially interacting drug pairs included those involving warfarin with nonsteroidal anti-inflammatory drugs (NSAIDs) and warfarin with thyroid hormones ( Table 4). The Appendix provides the number and percentage of prescribers who wrote prescriptions for each of the drugs of interest, the total number of prescriptions written by all prescribers for each drug of interest, and the median number of prescriptions written for each drug for prescribers who wrote at least 1 prescription for the medication of interest during the study period.
Results of the random-effects negative binomial model indicated that adjusting for total number of prescriptions pre-   = 2,204. An interaction term between treatment group and time period was not significant at the P < 0.1 level and therefore not included in the final model. C-statistic = 0.969. b Baseline period was the 12 months prior to a specified index date for each prescriber. The index date was the date when the PDA device was deployed to the provider in the PDA group or the date on which a provider in the comparison group registered to obtain the PDA. Follow-up period was the 12 months following the specified index date. CI = confidence interval; DDI = drug-drug interaction; PDA = personal digital assistant. medication history updates did not change the results (data not shown).

Evaluation of a Wireless Handheld Medication Management Device in the Prevention of Drug-Drug Interactions in a Medicaid Population
The random-effects logistic model used to assess whether the likelihood of prescribing at least 1 potentially interacting drug pair was associated with use of the wireless handheld PDA device (Table 6) produced results similar to the aforementioned negative binomial model. After adjusting for other factors, the likelihood of a provider prescribing at least 1 potential DDI combination was not associated with use of the PDA device; the interaction term between treatment group and time period was not significant at the P < 0.10 level and therefore dropped from the final model. After adjusting for other factors, rural providers were more likely to prescribe at least 1 potential DDI combination compared with urban providers in both time periods (odds ratio [OR] = 1.93, 95% CI = 1.25-2.96, P = 0.003). Internal medicine providers were more likely than family practice providers to have at least 1 potential DDI in both time periods (OR = 1.66, 95% CI = 1.17-2.36, P = 0.004). The proportion of providers with at least 1 potential DDI was not statistically significantly lower in the follow-up time period compared with the baseline period, regardless of treatment group (OR = 0.83, 95% CI = 0.69-1.00, P = 0.054). Again, stratifying the PDA group by frequency of medication history updates did not change the results (data not shown). The results of the models did not change with respect to the effect of the medication management program on the rate of potential DDIs when prescribers with pediatric and psychiatry specialties were excluded from the analysis.
The main focus of the present study was whether the provision of a nearly real-time medication history resulted in a reduction in potential DDIs. The handheld device did not provide e-prescribing or interaction alerts during the entire study period, but new functions were continually added. Today, however, such medication management systems routinely include e-prescribing, and many also support CDS. Drug interaction CDS has the potential to improve providers' recognition of clinically significant DDIs, although too many alerts, indicative of poor signal-to-noise ratios, may prevent clinicians from optimizing the information presented to them. 13 Alert fatigue occurs when clinicians are overwhelmed by the volume of alerts presented to them, in which case they begin ignoring both relevant and irrelevant alerts, potentially negating the safety benefits of the CDS system. 31 Many studies have documented discontent with alerts believed to be repetitive and irrelevant. [32][33][34][35][36] A number of other studies have demonstrated that clinicians frequently override DDI alerts. [37][38] For example, in a retrospective analysis of alerts generated by an ambulatory e-prescribing system, clinicians accepted only approximately 9% of drug interaction alerts presented to them. 38 Furthermore, clinicians were more likely to override an alert if the patient had already received the medication associated with the alert.
There are a number of potential explanations for the lack of reduction in potential DDIs associated with the wireless handheld health information system. The stand-alone system for only a portion of the prescribers' patients may explain the lack of effect seen in this study. No data were available at the prescriber level on the proportion of patients with Medicaid versus other insurance coverage. While PDA users were more likely to see Medicaid patients than the comparison group (as a requirement to be offered the device), the minimal number of times that the vast majority of prescribers routinely updated patient histories over the course of the observation period is indicative of the usefulness of the device.
Another potential reason no effect was seen was that a small subset of all potential DDIs were chosen to evaluate the application. Although these potential DDIs were chosen, in part, due to widespread use, therapeutic importance, tendency to produce harm, and clinical evidence, it is unknown if prescribers were aware of these interactions and if alerts were examined when e-prescribing functionality was added. Studies have indicated a considerable degree of discordance among drug compendia regarding the inclusion of DDIs and their associated severity ratings. 39,40 In addition, appropriate clinical responses for the targeted drug pairs may have included actions that would have still allowed the 2 agents to be co-prescribed. Such actions may have included counseling, monitoring, or temporarily stopping one of the medications. A prime example involves

■■ Discussion
This study suggests that availability of a stand-alone wireless handheld medication management application may not reduce the incidence of certain potential DDIs within a Medicaid population. One of the primary reasons that this device did not have a significant impact is the apparent low use of the device based on the number of times prescribers "updated" the medication history, suggesting the device was not frequently used. Integration of the device into day-to-day practice may have been difficult because providers often see patients with a variety of health insurance coverage types. This device provided information only on medications from Medicaid patients. Medication management programs that include data from only a single payer may be of limited value in preventing serious adverse drug events for patients of a given prescriber.
In a recent review of the role of computerized decision support, Baysari et al. (2011) lamented that knowledge deficiency is the most common factor contributing to prescribing errors. 23 In a review of prescribing errors within institutional settings, Tully et al. (2009) concluded that knowledge-based prescribing mistakes, including failure to account for patient comorbidities, commonly occur. 24 Given prior research in the area of health information exchange and medication safety, the present study results are not entirely surprising. A systematic review of the effect of e-prescribing on medication errors found that the greatest evidence of safety benefits with these technologies has been demonstrated with studies utilizing homegrown systems, studies that compared handwritten prescriptions with those electronically prescribed, and those studies involving chart reviews of prescription orders. 20 The current study did not involve any of these features. However, this study adds insight into the pragmatic use and effectiveness of a medication management device in averting potential DDIs across a variety of provider specialties and practice settings.
Previous research has documented the lack of complete medication history during medical office visits, 25 but data on the impact of health information exchange are limited. 26 Daniel et al. (2010) evaluated the role of medication history information to patients presenting to the emergency department and found shorter visits and lower cost of care when medication history was available. 27 Several studies in ambulatory settings have demonstrated safety benefits associated with more comprehensive CDS systems in CPOE applications. [28][29][30] Implementation of specific DDI alerts for warfarin in a health maintenance organization's electronic medical records system was associated with significant declines in prescribing rates for the interacting pair. 30 More recently, implementation of a commercially available e-prescribing system in an ambulatory setting resulted in a significant reduction in prescribing errors. 28 We could not find a published study that evaluated a similar early-generation wire-

Evaluation of a Wireless Handheld Medication Management Device in the Prevention of Drug-Drug Interactions in a Medicaid Population
use of it. 43 The most common reasons prescribers reverted back to written prescriptions were technical and software problems as well as time constraints. 43 In the present study, the reasons for return of the wireless handheld device within 365 days by 303 prescribers were not captured.
Sixth, the generalizability of this study may be somewhat limited due to a number of factors. The prescribers who participated in this study were all contracted with 1 particular state Medicaid program. The medication prescription records that served as the basis of this analysis were those of Medicaid beneficiaries, which may indicate greater likelihood of overthe-counter medications in prescription records than would otherwise be expected in commercial or Medicare populations. The limited number and type of drug interactions may further limit the generalizability of results. Seventh, the finding that prescribers specializing in pediatrics and psychology were less likely to prescribe interacting drug pairs is likely related to the drug pairs that were chosen for this study. Many of the DDIs selected for study inclusion contain medications commonly prescribed in adults with a variety of health problems, decreasing the likelihood that pediatric or psychiatry specialists would prescribe both potentially interacting medications for a given individual.
Finally, only potential drug interactions were evaluated. This study did not evaluate whether actual adverse drug events resulted from the potential DDIs identified nor did it evaluate prescribers' responses to alerts that they may have received about potential DDIs. The features of the wireless application employed in this study may not be generalizable to similar applications. For example, the prescription drug history accessible by prescribers was limited to 100 days in this program.

■■ Conclusions
Provision of a medication management program via a wireless handheld device with access to 100 days of patient-specific drug history by prescribers was not associated with a significant reduction in the rate of prescribing of clinically important DDIs. It appears that although e-prescribing and DDI checking functionality were added, prescriber use of the device waned over time and may have contributed to the lack of an effect. the rifampin-oral contraceptive drug pair. Consequences of concomitant administration of these 2 agents include risk of pregnancy, which can be avoided with the use of alternative barrier contraceptives without stopping either medication. In a study of the reasons providers commonly override DDI alerts, one of the most commonly cited reasons was "patient being monitored." 41 Furthermore, the present study involved a retrospective analysis of paid pharmacy claims. Therefore, there were no opportunities to examine prescribers' initial responses to DDI alerts. The paid claims represent final prescribing decisions along a continuum, beginning with prescriber order generation and terminating with the dispensing of a medication, most likely by a pharmacist. This continuum allows for multiple opportunities for a prescription to be changed or altered in response to a number of variables, including, but not limited to, DDI alerts arising from either the clinician or pharmacy CDS system.

Limitations
In addition to the retrospective nature of this study that prevented analysis of initial responses by prescribers to DDI alerts, the study design was quasi-experimental, difference-indifference; therefore, factors other than the use of the handheld device may have contributed to the observed results. The comparison group was selected from prescribers who requested the device but were excluded because of low Medicaid prescription volume or geographical distance from locations where training was provided on use of the device. Thus, the comparison group is not identical to the intervention group, but the impact of such differences is somewhat negated by the use of multivariate regression models controlling for confounding factors such as prescription volume. Third, further complicating the analysis is that the technology and capabilities of the systems evolved over time, making it difficult to isolate the effect of any one component of the device. In addition, drug information is dynamic, and the knowledge bases that serve as the foundation of CDS systems must constantly adapt to new information. Because this study spanned several years, it is possible that the timing of an individual's index date may have influenced the results. Fourth, another factor that was not controlled for was the disease severity of patients. However, many threats to the validity of our analysis were overcome through the pre-intervention/ post-intervention study design (i.e., each provider served as his/her own control).
Fifth, a number of unforeseen and unmeasured factors may have influenced the use of the device and its effectiveness related to preventing potential DDIs. For example, prescribers' perceptions of a system's ease of use may influence the extent to which the system is used. Provision of a technology is necessary, but insufficient, to guarantee its use. 42 A study by Wang et al. (2009) found that physicians who believed their e-prescribing system was difficult to use were more likely to discontinue